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^yi  Ahulrnnl 

The  intcrconncetkMi  of  •  Intfe  number  of  proceMors 
end  other  devices  to  form  n  pnrnllel/distributed  comput¬ 
ing  qrstem  is  n  resenrch  aren  receiving  s  great  deu  of 
attention.  One  method  is  to  use  a  multistage  network. 
This  paper  compares  two  classes  of  multistage  aetworks 
by  esaminiag  two  representative  networks:  the  General- 
ised  Cube  and  the  Augmented  Data  Manipulator.  The 
two  topoktgics  are  compared  using  a  graph  theoretic 
approach.  Dy  interpreting  the  graimical  representations 
<N  the  networks  in  different  ways,  different  implementa¬ 
tions  result.  The  cosU  of  the  various  implementations 
are  compared  taking  ^ftSi  considerations  into  account. 
Finally,  the  robustness  of  tM  different  networks  is  meas¬ 
ured  and  contrasted.  v 

L  introduetton 

The  interconnection  of  a  larM  number  of  processors 
and  other  devices  to  form  a  parallel/distributed  comput¬ 
ing  system  is  a  research  area  receiving  a  great  deal  of 
attention.  Many  different  approaches  to  the  interconnec¬ 
tion  method  have  been  proposed  and  discussed  including 
the  use  of  buses  |4S|.  hierarchies  of  buses  |30|,  direct  links 
1 10),  single  staM  networks  (11^,  multistage  artworks  (7, 
17,  27,  3^,  and  croasbais  (4^.  An  impiwtant  aspect  of 
this  rcaearch  is  the  evaluation  and  comparison  of  ue  pro¬ 
posed  approaches  |4,  Iff,  3S,  40).  The  conclusion  most 
often  reached  is  that  the  best  scheme  to  use  in  a  particu¬ 
lar  design  highly  depends  upon  the  intended  application, 
perfomranec  reguirementa,  and  coat  constraints.  Once  a 
connection  metliod  is  chosen  (c.l  single  stage  network),  a 
specBc  design  must  be  decided  upon  and  then  imple¬ 
mented.  During  this  phase  of  a  syrtem's  specilcMion,  it 
is  important  m  the  derigner  to  nndersiM  fully  the 
differences  and  similarities  between  candidate  dmigns. 
This  paper  is  an  investigation  of  two  classes  of  multistage 
aetworks  that  have  been  considered  for  use  ir  a  numbw 
of  nratems.  In  pnrtieular,  this  work  is  part  of  a  network 
evMuatioo  study  for  the  PASM  |Sff|  and  PUMPS  sys¬ 
tems. 

The  Generalised  Cube  and  Augmented  DaU  Maai- 
pulator  (ADM)  networks  art  defned  in  Sectioa  II.  Their 
relation  to  other  multistate  networks  dsKribed  in  the 
literature  is  also  discussed.  Using  graph  theon,  the  net¬ 
works’  topologies  win  be  compared  m  Section  lu.  In  Sec¬ 
tion  IV,  two  Hnplemcntntiono  resulting  from  two  diShrent 
graph  intsrpretations  will  ho  ciamined  to  compare  the 
cost  of  earn  network.  Here,  using  VLSI  chips  is  con¬ 
sidered  and  costs  arc  compared  relative  to  the  fraction  of 
a  stage  that  can  he  implemented  on  one  chip.  Finally, 
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Figure  I:  Generalised  cube  network  Isr  Nwg.  The  Imr 
legitimate  states  of  an  interchange  bos  nre  Awwn. 


Section  V  will  contain  as  analaysis  of  the 
network  exhibits. 


n.  The  Oeneeallnad  Ouho  nwd  ABM  Watworks 

The  CSeneraliied  Cube  neterork  is  a  multistag-  be 
type  network  bipokigy  that  was  introdueed  as  a  e^ndard 
for  comparing  network  lopologica  |34).  Assume  the  act- 
w(wk  has  N  inputs  sad  N  outputs:  in  Fin.  I,  N-8.  The 
tieneralizcd  (’uhe  topology  hm  nwiog^  N  stages,  where 
each  stage  rt»nsists  of  a  set  of  N  lines  connected  to  N/l 
interehange  boxes.  Each  interchange  hm  is  a  tw»>input, 
two-nutput  device.  The  labels  of  the  inut/ontpnt  fiacs 
entering  the  upper  and  lower  inputs  of  an  interehange 
box  serve  as  the  labels  for  the  upper  and  lowsr  outputs, 
respectively.  Each  interchange  him  can  ha  aot  to  me  of 
the  four  b-gilimate  states  shorn  (IT). 

The  ronnections  in  this  nnwork  are  baaed  m  the 
cebe  inlermnnertion  functiom  |34|.  Lot  P^Pa-j  ...  P|Po 
be  the  binary  representatim  of  an  arMtmfT  I/O  hae 
label.  Then  the  n  cube  intercmnectim  ftoctnm  cm  be 
defined  as: 

f'HPs  I  -  PiPo)  *  Fu-i-Fieiiil-i-FiRl 
where  0<i<n,  0$P<N,nndRdmo>mtht<ogylimmtof 
Pi.  ThiT  means  that  the  CUM|  htcrcomactim  Ametion 
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STAGE  2  I  0 

Figure  2:  Augmeutcd  daU  manipulator  network  for  N=8. 
(Lowcrrauc  letters  represent  end>around  connections.) 

connects  P  to  cubcjlP),  where  cubi^P)  is  the  I/O  line 
whose  label  differs  from  P  in  just  the  i"  bit  position.  Stage 
i  of  the  GeneraJued  Cube  tc^logy  contains  the  cube  inter¬ 
connection  function.  That  w.  it  pairs  I/O  lines  that  differ 
in  the  i**  bit  position. 

The  ADM  network  is  shown  in  Fig.  2  for  N=8.  It  is 
based  on  Feng’s  data  manipulator  |I2|.  In  this  network,  a 
sfsfe  consists  of  N  $witelum§  tl*mtnt$  or  nedes  and  the  3N 
data  path^  that  are  connected  to  the  inputs  of  a  succeed¬ 
ing  stage. '  Each  node  can  connect  one  of  its  inputs  to  one 
or  more  of  its  outputs.  At  stage  i  of  the  ADM  netsrork, 
0<i<o,  the.first  output  of  node  j  is  connected  to  the  input 
of  node  (j-i'f  mod  N  of  the  next  stage;  the  second  output 
is  COOK',  -ted  '  o  the  input  of  nude  j;  and  the  third  output  b 
ctiimccted  to  the  input  of  node  (j'f  2*)  mod  N.  BMausc 
(j''2* ')  e<|uab  (j'h2*'')  mod  N,  there  are  actually  only 
two  distinct  data  paths  instead  of  three  from  each  node  in 
stage  n-1  (in  the  Igure,  stage  2).  There  is  an  additional 
set  of  N  nodes  at  the  output  stage. 

A  number  of  systems  have  been  proposed  and/or 
built  that  use  multistage  networks  |e.g.  8,  6,  19,  SI, 
Among  the  networks  that  have  bMu  proposed  are  the 
ADM  TSSl,  bsMline  (481,  binary  n-cubc  t27],ulata  manipu¬ 
lator  1 12],  delta  t26|.  Gamma  |2^,  Generatim  Cube  [S4], 
inverse  ADM  Jss),  omega  (IT],  StARAN  flip  (71,  and  SW- 
banyan  ll^.  atudics  have  shown  that  the  basenne,  binaiy 
n-cube,  Generalised  Cube,  omega,  STARAN  flip  and  Sw- 
baayan  (SsFsg)  neiworu  are  all  topologically  equivalent 
(28,  82,  37,  48].  Differences  between  these  networks  are 
due  to  proposed  coatiol  whemes,  whether  or  not  a  broad¬ 
cast  capaMHty  is  included,  and  the  method  used  to 
number  input  and  output  ports.  All  of  these  networks 
bdong  to  tne  generni  enus  of  cube  type  networks.  This  in 
turn  M  in  the  darn  of  banyan  networiis.  They  can  also  be 
considered  delta  networks  since  each  can  be  controlled 
uaiag  one  digit  of  a  control  vector  (or  number)  per  stage. 
Because  of  the  similarities  among  these  networks,  a 
designer  is  not  faced  with  cboasiog  bdween  seven  different 
networks, -rather  the  choice  is  whether  or  not  to  use  a  cube 
type  network. 

The  data  manipulator,  ADM,  lADM,  and  Gamma 
networks  am  lopongically  identical.  The  differences 
betwenn  thaw  networks  am  the  control  Mhcme,  wder 
ilofm  am  tmanversud  and  switch  complexity.  11w 


switches  in  each  stage  of  the  data  manipulator  are  divided 
into  two  groups.  Each  group  receives  an  independent  set 
of  control  signals  and  all  switches  in  a  group  respond 
identically.  Each  switching  element  of  the  ADM,  lADM, 
and  Gamma  networks  is  controlled  individually.  The 
stages  of  the  lADM  and  Gamma  networks  are  traversed  in 
an  order  opposite  to  that  of  the  ADM  and  data  manipula¬ 
tor.  Also,  the  Gamma  network's  switching  elements  are 
3x3  crossbars  (as  oppo^  to  selecting  one  input  at  a  time). 
None  of  the  networks  is  a  member  of  the  banyan  or  delta 
classes.  The  capabilities  of  the  Gamma  network  are  a 
superset  of  the  ADM  and  lADM  networks.  It  has  been 
shown  in  turn  that  their  capabilities  are  a  superset  of  all 
the  cube  type  networks  as  well  as  the  data  manipulator 
network  (OT|.  Data  manipulator  type  networks,  however, 
are  more  complex  than  cube  type  networks.  For  example, 
there  are  multiple  paths  between  all  nontrivial 
source/destination  pairs  (i.e.  source  addrem  ^  destination 
address).  The  redundancy  provides  a  degree  of  fault  toler¬ 
ance.  A  common  feature  of  all  cube  type  networks  is  that 
there  is  exactly  one  path  through  the  network  for  each 
source/destination  pair.  This  property  makes  control 
schemes  simple  but  any  single  failure  of  a  link  or  switch 
will  disaltow  the  use  of  any  path  requiring  the  failed  com¬ 
ponent. 

Thus  there  exists  the  classic  tradeoff  between  cost 
and  performance  when  choosing  between  the  two  network 
types.  In  this  paper,  one  representative  network  from  each 
type  will  be  compared:  the  Generalised  Cube  and  the 
ADM.  Both  networks  have  the  same  number  of  input  and 
output  ports  and  individual  switching  element  control. 
Routing  tag  schemes  are  available  for  the  networks  (17, 
24,  33,  34],  so  it  is  assumed  that  they  are  used  to  imple¬ 
ment  network  control. 

Some  aspects  of  the  Generalised  Cube  and  the  ADM 
networks  have  been  compared  elsewkerc.  The  ability  of 
the  ADM  network  to  permrm  all  ike  functions  a  Gcacral- 
ixed  Cube  can  was  demonstrated  in  (37].  la  (1|,  the  total 
number  of  unique  permutation  connections  sack  network 
can  perform  was  compared.  Tbk  paper  is  concerned  with 
comparing  cost  and  robustness  or  inherent  fault  tolerance. 
Cost  is  examined  from  two  points  of  view.  The  flrst  is  the 
common  method  of  counting  links  and  switching  nodes.  In 
this  case,  gr^  theory  with  a  consistent  interpretation 
(]^o  are  possible)  ■  used  to  insure  a  “fair*  comparison. 
The  second  point  of  view  b  oriented  toward  VLSI  con¬ 
siderations.  Modules  for  each  network  rsmring  rougbly 
the  same  number  of  pins  are  compared.  The  change  in 
relative  cost  b  also  examined  when  as  much  as  one  whole 
stage  b  placed  on  one  chip.  Robustnem  b  measured  by 
calculating  the  average  number  of  network  inputa  and  out¬ 
puts  affected  by  the  removal  of  a  aingb  link  or  switching 
element.  The  calculations  are  performed  for  both  of  the 
graph  interpretations  to  be  deflnM. 

ID.  Graph  Thsuryi  A  Common  Ground  fee 
Comparing  Notworhs 

Graph  theory  has  been  used  by  Goke  and  Lipovski  m 
the  basis  for  deiining  a  class  of  networks  called  banyans 
[1.8|.  The  graphs  usm  to  represent  these  networks  consbt 
of  netfes  connected  by  undimrfed  area  ^  deflnition,  in  a 
banyan  there  b  one  and  only  one  path  from  input  to  out¬ 
put  |I5|.  In  thb  paper  there  b  no  restriction  on  the 
number  of  paths  from  input  to  output. 

It  has  been  observed  in  [Iq  that  the  Generalised 
Cube  network  (Fig.  I)  hm  the  graphbal  representation 
shown  in  Fig.  3.  Thb  graph  also  represents  an  SW-baayaa 
(with  S-F=2).  The  graph  can  be  interpreted  a  number  of 
different  ways.  One  b  to  treat  each  node  (vertex)  (a  circb 
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Figure  3:  Graphicul  representatioa  of  the  Generaincd 
Cube  network  for  N=3. 
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in  the  ligiire)  as  a  switch  and  each  are  (edge)  (a  line  in  the 
ligitre)  :ts  a  link.  To  model  the  network's  Mavinr  under 
this  interpretation,  (he  switch  (node)  shown  in  Fig.  4a 
should  only  connect  one  of  the  input  links,  a  *r  b,  to  one  tA 
the  output  links,  c  or  d.  An  implementation  basH  on  this 
interpretation,  for  an  N  input/output  network,  would  con* 
sisi  of  ti+  I  stages  of  N  switches,  with  2N  lines  between 
9  stagisi.  'I'lie  TI(A('  reconflguraMe,  multimirroprocessor 
C  system  contains  an  SW'banyan  constructed  from  switches 
T  of  this  type  (hut  that  have  two  inciHning  and  three  out^ 
P  ing  links,  i.e.  S=2  and  F=3)  |2fM.  A  second  interpretatioa 
is  to  tri-at  the  nodes  as  links  and  the  arcs  as  forming  inter* 
'p  change  iMixes.  For  example,  the  thickened  lines  in  Fig.  3 
can  l»‘  considered  to  represent  the  interchanK  box  with 
inputs  2  and  0  (compare  this  to  Fig.  I).  In  this  case  the 
SW-banyan  implementation  would  have  the  same  struc¬ 
ture  as  specified  here  for  the  cube  jassuming  a  bidirec¬ 
tional  iii'twork).  This  interpretation  Is  illustrated  in  Figs. 
4l>  and  Ic.  i-^ach  of  the  arcs  labeled  a  through  d  in  Fig.  4b 
acts  as  a  entsspoint  switch  in  Fig.  4c.  When  viewed  this 
nay.  the  iHtrlion  of  the  graph  within  the  dashed  lines  of 
Fig  th  behaves  as  a  2x2  crowbar  or  interchange  box.  If  a 
and  d  are  "on,’  the  straight  setting  is  obtained;  b  and  c  on 
rorres|y>nds  to  exchange;  a  and  b  on  corresponds  to  upper 
bro.idcasl;  and  c  and  don  corresponds  to  lower  broadcast, 
('onlliri  occurs  if  a  and  c  or  b  and  d  are  on  at  the  same 
time.  A  third  |M»ssible  interpretation  of  the  graph  in  Fig.  3 
is  to  «^iiate  nodes  with  2x2  interchance  braes  and  ares 
with  links.  In  that  case.  Pig.  3  would  represent  a  site 
\  =  IR  Generalized  C^ube  network.  This  tntf:n>teiation  will 
not  be  di.scus.si>d  further  in  this  paper. 

The  graphical  representation  of  the  ADM  network 
(Fig.  2)  is  shown  in  Fig.  5.  Since  there  are  multiple  paths 
from  input  to  output,  this  is  not  a  banyan  graph.  This 
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Figure  4:  (a)  A  node  from  the  graph  representing  the  Gen¬ 
eralised  ^be  network.  When  equated  with  a  switch, 
input  a  er  b  can  be  connected  to  output  c  or  d.  (b)  Four 
K^es  from  the  graph.  When  the  arcs,  a,  b,  c,  and  d,  are 
equated  with  switches,  a  SxS  cranAar  is  obtained,  (c)  The 
components  of  a  erosAar  that  corrwpond  to  the  graph  in 
(b). 


Figure  S:  Graphical  repressntalion  of  the  anysnted  data 
manipulator  fnr  Nsfi. 
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KigUK  6:  ImpIcmenUtioo  of  the  augmented  data  manipu¬ 
lator  for  N=8  when  the  graph  of  Fig.  S  is  interpreted  with 
arcs  equated  to  switches. 

graph  can  be  obtained  bv  adding  the  dashed  lines  shown 
to  the  graph  in  Fig.  3.  When  switcht-s  are  <M|iiated  with 
nodes,  the  network  depiction  in  Fig.  2  is  obtained.  When 
switches  are  equated  with  arcs,  the  network  looks  like  that 
sh«)wn  in  Fig.  6.  In  the  figure,  two  nodes  directly 
connected  by  a  solid  line  betwe4>n  stages  are  represented 
by  a  singb-  node  in  Fig.  S.  Note  that  the  labels  on  end- 
>ir  ■■Mid  c<  nnections  in  both  Fig.  .S  and  Fig.  0  are  attached 
to  .e  same  arcs  (links)  in  the  network.  This  second  type 
<if  iiiipieinentation  is  examined  in  (3M|,  where  I.SI  paekag* 
iagoi  network  building  blocks  is  discussed. 

Though  the  same  ADM  network  is  represented,  F^. 
3  and  6  kiuk  rather  different.  Depending  upon  which 
representatioa  k  chosen,  a  comparison  with  the  Generai- 
isH  Cube  in  Fig.  I  could  produce  different  conclusions. 
Comparing  Fi|^  I  and  3,  one  might  conclude  that,  in 
addition  to  having  an  extra  eoiumn  of  switches,  the  ADM 
has  twice  u  many  switching  nodes  and  three  times  as 
many  links  as  the  Generalised  Cube  network.  It  would  be 
easy  to  decide  that  the  ADM  network  is  considerably  more 
expensive.  On  the  other  hand,  comparing  Fip.  1  and  •,  it 
appears  the  only  difference  is  N  extra  links  that  intercon¬ 
nect  switches  within  each  stage  of  the  ADM  network.  The 
latter  comparison  is  more  accurate  because  the  network 
depictions  of  Fip.  1  and  6  are  based  on  the  same  interpre¬ 
tation  of  the  networks'  respective  graphs.  Thus  when 
making  comparisons,  it  is  important  to  either  compare 
graphical  representatioas  or  coosisteat  interpretatioas  of 
those  paphs.  In  the  next  section,  the  latter  is  done  for 
both  interpretatioas.  Thb  h  so  the  resulting  implementap 
tions  can  m  compared  as  well. 


IV.  Cost  Comparlaoa 

A.  /hlradurfim.  The  purpose  of  this  section  is  to  compare 
the  cost  of  the  Generalised  Cube  network  to  that  or  the 
ADM  network.  To  do  this,  implenientatkms  of  each  neb 
work  are  examined.  Since  two  bask  implementations  are 
possible  lor  each  network,  to  be  fair,  only  implementations 
corresponding  to  the  same  graph  interpretation  are  com- 
pared. 

A  //nrdwnrc  AeniiMfMns.  There  arc  two  ba^  wavs  to 
hnnleniett  multistats  networks.  They  can  be  cwcuit 
switched  or  packet  swHehed.  In  circuit  switching,  a  com¬ 
plete  path  is  sstabiished  from  input  to  output  and  must  be 


held  for  the  duration  of  the  comiminication.  Circuit 
switching  is  often  used  when  processors  are  connected  to 
the  netwtirk  inputs  and  memories  are  connected  to  the 
outputs.  Designs  for  circuit  switched  interchange  boxes 
have  been  discussed  in  |0,  33,  3d|.  In  packet  switching, 
messages  are  decomposed  into  pMkels  which  each  make 
I  heir  way  from  stage  to  stage  until  the  output  is  reached. 
'Phis  melhod  is  often  used  in  configurations  that  connect 
processing  element  |processor/inenH>ry  pair)  j  to  input  j 
and  output  j  of  a  unidirectional  network.  Packet  switchi-d 
netwi>rk  switching  element  designs  have  been  discussed  in 
lll.33.llj. 

In  the  remainder  of  this  paper,  implementations  will 
be  discusst-d  primarily  in  terms  of  packet  switching, 
(‘ircuit  switched  versions  can  be  obtained  by  replacing  My 
ipieues  shown  with  busses.  Other  thM  this,  remaining 
differences  are  in  the  control  logic,  however  the  logic  b 
shown  only  at  the  block  diagram  level.  Only  key  elements 
of  the  implementations  to  be  discussed  are  included  since 
many  variations  of  the  bask  designs  are  powible.  For 

more  detail  si-e  |l  1, 33,  d!l- . . 

C.  Ctitertliui  Cute.  Pig.  7  shows  two  designs  for  a  Gen¬ 
eralised  Cube  switching  element.  Fig.  7a  results  when 
switches  are  cviuated  with  nodes  in  the  graph  (thb 
corres|Mtnds  to  Figs.  4a  Md  3).  One  of  the  two  inputs  b 
selecira  depending  on  the  requests  (if  My)  received  by  the 
(left  half  of  the)  control  logk,  whkh  b^les  My  needed 
arbitration.  A  single  output  link  b  shown,  but  it  b  to  be 


Figure  7:  Implementation  of  Genemliied  Cube  switches, 
(a)  node  w  switch  interpretation,  (b)  are  s  switch 
intorpretatioa. 
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Figure  8:  Four  switche*  from  Pig.  7s  combined  to  form  one 
switch  (within  dashed  lines)  equivalent  to  that  in  Fig.  7b. 


connected  to  two  other  switches  as  shown  in  Fig.  8.  A  bit 
in  the  routing  tag  is  examined  by  the  controi  logic  which 
then  determines  to  which  switch  a  request  for  access 
should  be  made.  The  (right  half  of  the)  control  logic  main¬ 
tains  the  queue,  interprets  the  routing  tag,  generates 
access  requests,  and  receives  grants  for  access  requests. 
Switches  that  implement  nodes  in  column  3  of  Fig.  3  only 
contain  hardware  to  the  right  of  the  dashed  line  in  Fig.  7a. 
Switches  that  implement  column  0  nodes  onlv  conimn 
hardware  to  the  left  of  the  dashed  line.  A  detailed  design 
of  this  type  is  discusaed  in 

If  ares  in  the  graph  are  equated  with  switches,  then 
four  arcs  form  a  2x2  crossbar  or  interchange  box  (see  Fi^. 
4b,  ic,  and  I).  An  implementation  for  this  is  shown  in  Fig. 
7b.  Here  two  input  queues  are  required.  As  long  as  a 
given  queue  is  not  full,  incoming  packets  for  that  queue 
will  be  accepted.  Logic  is  require  to  handshake  with 
other  interchange  boxes,  maintain  two  queues,  and  inte^ 
prel  the  routing  tags  at  the  head  of  each  queue.  This 
only  interprets  the  tags  in  order  to  request  the  desired  set- 
tinp  for  the  multiplexers.  Logic  associated  with  the  mul¬ 
tiplexers  performs  any  necessary  arbitration.  It  also 
makes  appropriate  requests  of  other  interchange  boxes 
once  the  multiplexers  are  set.  Different  protoi^  and 
design  variations  for  this  type  of  switching  element  are  dis¬ 
cuss^  in  |28|.  The  performance  of  networks  implemented 
with  these  interchange  boxes  has  been  studied  in  (1 1,  21). 

The  equivalence  of  two  networks  implemented  with 
the  two  kinds  of  switching  nodes  is  illustrated  in  Pig.  8. 
Four  of  tlw  switching  elements  shown  in  Fig.  7a  are  con¬ 
nected  as  prescribed  by  the  papb  in  Fig.  3.  It  can  be  seen 
that  the  hardware  within  the  dashed  lines  is  Mentical  to 
that  shown  for  the  interchange  box  in  Pig.  7b.  The 
handshaking  lines  (directed  daBhed  lines)  shown  connect¬ 
ing  control  units  are  equivalent  to  internal  connectiOBs 
between  the  tag  interpretation  and  queue  control  io^  and 
the  arbitration  and  output  request  hinc  in  the  contra  unit 
of  Fig.  7b.  It  is  thus  sroarent  that  the  same  total  amount 
of  hardware  b  required  for  either  implementation,  but  that 
the  two  naph  interpreUthms  lend  to  different  network 
building  bloeks  or  pnekaginis  for  the  components. 

D.  Aufmenfed  Dafn  htvnit&Mltir.  Two  implementations 
for  the  ADM  network  are  mktwb  in  Fig.  9.  Fig.  9a  results 
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Figure  9:  Implementalioa  of  augmented  data  manipulator 
switches,  (a)  node  ~  switch  interpretatioa.  (b)  arc  s 
switch  interpretation. 


from  equaling  the  nodes  of  Fig.  5  with  switchea.  In  thb 
design,  the  multiplexer  selects  from  among  three  input 
links  and  the  output  link  b  connected  to  three  otW 
switches.  The  control  signab  shown  on  the  output  side  in 
Fig.  9a  are  used  to  determine  which  of  the  twitches  b  to 
read  the  data  from  the  output  link.  A  broadcast  b  per- 
ffwmed  by  selecting  more  than  mie  switch.  The  baric  rout¬ 
ing  lag  .scheme  for  the  ADM  network  (24)  raMircs  the 
rouliiig  lag  logic  to  examine  two  bits,  so  it  b  slightW  more 
complex  than  I  hat  required  in  the  CieneraNfed  Cube.  As 
wilh  I  he  (Generalised  Cube,  the  twitches  implementiag 
nodes  in  columns  0  and  3  of  Fig.  S  only  remire  the  logic  to 
the  left  and  right,  respectively,  of  the  dashed  line  ia  Fig. 
9a. 

If  arcs  are  equated  with  switchea,  aa  ImMemeatation 
similar  to  Ihe  interchange  box  b  obtained  aa  thowa  ia  Pig. 
9b.  Here,  however,  the  outputs  from  the  queuea  must  be 
conned ed  to  multiplexers  in  two  other  twMehiag  elemeala 
(as  shown  in  Fig.  9)  via  infro-tiafc  luaact.  SimUafly,  the 
two  multiplexers  shown  here  mutt  accept  eoanectioac 
fr«>m  Hie  queues  of  two  other  switching  eleawata.  Two 
control  Kign.-ils  must  also  aecompaay  each  of  the  intra- 
stage  busses. 

E.  Compenten.  An  appmimale  eoat  eomparboa 
between  the  Generalised  Cube  aad  the  ADM  Mtwoik  can 
be  made  bv  comparing  their  respective  awitehiag  sismsats. 


«l« 


SiMt  tlw  ckoice  it  arbitrary,  Figt.  7a  and  Oa  will'  be  eom- 
parad.  Botb  ra<|airt  a  tingle  aueue.  If  the  coat  of  the 
OHOiie  and  ita  aaaoeiated  control  logic  dominates  the  cost  of 
Ute  twitching  eieiiimt,  then  the  ADM  switch  wHI  cost  only 
slightly  more  thaa  a  Generalised  Cube  twitch.  Oa  the 
otbw  aaad,  lor  a  circuit  switched  implementation,  the 
multipleier  and  control  logic  in  an  ADM  switching  ele¬ 
ment  will  cost  about  Vfii  more  than  that  requirco  in  a 
Gencraliied  Cube  twitching  element. 

The  penpective  changes  somewhat  when  implement* 
iag  these  four  designs  in  VLSI  b  considered. 
Input/Output  (I/O)  re^uircmenU  and  logic/pin  ratio 
become  importMt  considerations.  For  constructing  a 
Generalised  Cube  network,  the  interchange  bos  in  Fig.  7b 
b  a  better  choice  thaa  the  switch  in  Fig.  7a.  The  inter* 
change  hoi  (Fig.  7b)  hat  approximately  33%  more  pins 
but  approximately  100%  more  logic  than  the  switch  (Fig. 
7a).  For  the  ADM  network,  the  logic/pin  ratio  b  nearly 
the  same  for  both  of  the  designs  in  Fig.  0.  The  des^  in 
Fig.  Ob  hat  approximately  twice  as  many  pins  and  twice  at 
much  lo|^  as  that  shown  in  Fig.  Oa.  The  extra  links  that 
give  the  ADM  network  its  superior  capabilities  over  the 
Generalised  CuIm  reipiire  a  larger  number  of  pins  on  the 
VLSI  chipe  being  considered. 

The  design  of  Fig.  7b  and  that  of  Oa  have  approxi¬ 
mately  the  same  number  of  pins.  If  thb  number  of  pint 

(due  to  the  data  path  width)  b  near  technologica)  limht 
and  thus  the  design  of  Fig.  Ob  will  not  It  one  chip),  then 
the  Generalised  Cube  interchange  box  b  superior  aue  to 
the  logic/pin  ratio.  Assuming  the  cost  of  two  chips  with 
the  same  number  of  pins  b  about  the  same,  an  ADM  net¬ 
work  would  be  more  than  twice  as  expensive  as  a  General¬ 
ised  Cube  network  of  the  same  sise  (when  realised  with 


these  two  rwpective  chips).  The  logic/pin  ratio  of  the 
ADM  chip  (Fig.  Oa)  can  be  improved  considerably  by 
ii  -plementing  extra  eapabilitkt  the  ADM  netwrork  b 
Lo  to  support  (23,  2t|.  These  capabilities  include 
dynamic  rerouting  of  blocked  messagra  and  stage  look* 
anead  with  rerouting  for  blockage  prediction.  None  of  the 
additional  features  requires  any  extra  pins.  The  additional 
capabilities  are  possible  because  of  the  extra  paths 
between  input  ana  output  and  thus  are  not  availaUe  for 
the  Generalised  Cube  network. 

As  advances  in  packaging  technolun  continue,  the 
cost  diisrence  between  the  Gencralisea  Cube  and  the 
ADM  will  narrow  considerably  until  the  ADM  b  more 
cost-effective.  To  see  thb,  examine  Fig.  0.  The  larger  the 
number  of  switching  elements  (of  the  type  in  Fig.  9b),  in 
the  same  stage,  that  can  be  placed  on  a  sinnle  chip,  the 
more  intra-stage  busses  can  be  internalised.  This  rrauces 
the  I/O  overhead  of  the  extra  links.  If  a  whole  st^e  can 
be  placed  on  one  chip,  then  the  ADM  network  requires  the 
same  number  of  chips  and  connectkms  between  chips  as 
the  Generalised  Cube  network.  The  assumption  here  b 
that  the  chip  circuit  density  b  not  suflicient  to  support  a 
crossbar  but  it  will  accommodate  nuire  k»c  than  one 
stage  of  a  Generalised  Cube  requires.  The  ADM  network’s 
structure  thus  flib  a  gap  between  the  cube  type  networks 
and  crossbars.  Until  very  large  portions  of  an  interconnec¬ 
tion  network  can  be  placed  on  a  single  chip,  it  b  clear  that 
the  ADM  network  will  be  more  expensive  to  implement 
than  the  Generalised  Cube,  thou^  the  difference  will  con¬ 
tinue  to  decline.  Thus,  it  b  important  to  determine  the 
networks'  cost-effiMtiveness.  It  hsa  already  been  pointed 
out  that  the  ADM's  capabilities  are  a  superset  of  the  Gen¬ 
eralised  Cube's.  Another  factor  that  b  becoming  more 
important  as  the  eonstruction  of  enurnKius  systems  b  con¬ 
sidered,  will  be  dbeussed  in  the  next  section:  robustness  or 
inherent  fault  tolerance. 


V.  Robnetaoesi  A  Comparison  Of  Degradation 
Under  Component  Failure 

In  thb  section,  the  robustness  of  each  network  b 
measured  by  removing  a  single  component  (lidk  or  switch) 
and  counting  the  number  of  input  and  output  ports  th^ 
arc  affected.  An  input  port  b  considered  affected  if  it 
cannot  send  a  messa^  to  all  output  ports.  An  output  port 
is  considered  affected  if  there  w  at  least  one  input  port 
from  which  it  cannot  receive  messages.  Since  the  number 
of  ports  affected  varies  with  the  location  of  the  removed 
link,  averages  are  computed. 

The  average  number  of  affected  ports  b  calculated  for 
botb  implementations  of  each  network.  These  calculations 
are  performed  using  turo  different  rules  for  counting 
affected  ports.  The  first  rule  requires  all  I/O  ports  to  be 
consider^.  Under  thb  rub,  it  has  beea  shown  that  some 
permutation  connections  can  be  routed  around  a  faulty 
link  in  the  ADM  network,  but  thb  b  not  true  in  generu 
|33|.  The  second  rub  aibws  "severely*  affected  ports  to  be 
dbabled  and  thus  not  included  in  the  count.  Thb  rule 
takes  into  account  a  practbal  system  response  to  a  net¬ 
work  fault:  the  disabling  of  some  components  so  that 
operation  can  continue,  but  in  a  degraded  mode.  Thb  b 
feasible  if  the  neturork  b  used  for  asynchronous  communi¬ 
cation  by  cooperating  processors  (MIMD  mode).  If  the 
network  b  used  in  a  vnehronous  mode  to  perform  permu¬ 
tation  connections  (SIMD  mode)  dbablmg  some  com¬ 
ponents  b  not  feasible.  Thus  the  following  analysb  applies 
only  to  use  in  MIMD  environments,  'nie  second  rule  b 
implemented  ss  follows.  Referring  to  the  graplw  in  Fip.  3 
and  S,  if  a  straight  (or  horisontal)  arc  at  level  j  b  removed, 
then  input  port  j  and  output  port  i  are  disabled.  If  links 
are  equated  with  arcs,  one  paw  of  I/O  ports  b  dbabled.  If 
switches  are  equated  with  arcs,  since  two  straight  arcs  are 
included  in  each  switching  element  (F^  7b  and  9b),  two 
pairs  of  I/O  ports  are  dbwbd.  Thus,  in  Figs.  1  and  9,  the 
I/O  ports  whose  addresses  correspond  to  the  output  labeb 
on  a  given  switching  ebment  arc  disabled  if  that  switching 
element  fsib. 

The  results  using  the  first  rub  are  shown  in  Table  1 
and  using  the  second  rub  are  in  Tabb  D.  The  derivations 


Table  I:  Average  number  of  affected  I/O  ports  in  the  Gen¬ 
eralised  Cube  and  ADM  networks  when  links  and  switches 
are  removed.  All  ports  aro  considered.  Node^switch 
impbincntatkm  corresponds  to  Fip.  3  and  2.  Arc=switeh 
implementation  corresponds  to  Figs.  1  and  6. 
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Tabb  II:  Average  number  of  affected  I/O  ports  in  the  Gen¬ 
eralised  Cube  and  ADM  networks  when  links  and  switches 
are  removed.  Severely  affected  ports  are  dbabled  and  not 
counted. 
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of  all  the  rcsuUa  tabulated  here  are  too  lengthy  to  include, 
but  can  be  found  in  [20].  A>  an  example  of  how  the  valuox 
ve  calculated,  conaider  the  cane  of  a  link  failure  in  the 
ADM  network,  implemented  by  equating  nodes  with 
switches.  A.<«ume  the  link  is  in  stap  i  (see  Fig.  2).  it  the 
link  is  non>straight,  none  of  the  I/O  ports  are  all^ti^ 
since  the  routing  tag  scheme  in  |24|  can  dynamically 
reroute  to  another  path  that  does  not  use  that  link.  For 
example,  a  path  from  input  1  to  output  7  is'-f  -f  2', 
straint,  which  uses  the  -t‘2'  link  in  stage  I,  between 
switAes  5  and  7.  if  that  link  is  bad,  the  message  can  route 
straight,  -2',  straight  and  avoid  it.  If  thq  bad  link  is  a 
straight  link  at  level  j,  then  there  are  2* input  ports 
that  cannot  send  a  message  to  output  port  j  (namely  those 
ports  whose  low  order  i  bits  of  their  addresses  agree  with 
j's|.  In  this  case,  output  port  j  is  the  only  output  port  that 
cannot  receive  mess^es  irom  all  input  ports.  This  fart  is 
a  result  of  the  properties  derived  in  (24l.  For  example, 
suppose  the  straight  link  in  stage  1,  level  4  (in  Fig.  2)  is 
bad.  Output  4  will  be  unable  to  receive  messages  from 
inputs  0  and  4.  On  the  other  hand,  all  the  other  output 
pwts  are  unaffected.  Even  though  the  4-  2*  straight,  -I-  2* 
path  from  input  0  to  output  S  includes  the  bad  straight 
link,  a  message  can  simply  take  the  straight,  -2',  '2*  path. 
The  average  number  of  I/O  ports  affected  by  a  bad  straight 
link  is  calculated  as: 


I  S't  .  ,  I 
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Since  the  failure  of  a  4-  2*  or  a  -2*  link  does  not  affect  any 
I/O  ports,  if  link  failures  are  equally  likely,  then  the  aver* 
age  over  all  links  is  one  third  of  the  above  value. 

In  Table  I,  the  ratio  of  the  average  number  of  affected 
I/O  ports  in  the  Generalised  Cube  to  those  in  the  ADM  n 
computed.  Regardleas  of  network  sise,  in  the  implementa¬ 
tion  in  which  switches  are  equated  with  nodes,  a  link 
failure  in  the  Generalised  Cube  network  affects  «x  times 
as  many  ports,  on  the  average,  as  a  link  failure  in  the 
ADM.  For  all  the  remaining  types  of  failures  and  imple¬ 
mentations,  the  ratio  is  two. 

The  measurement  using  the  first  rule  is  a  very  conser¬ 
vative  indication  of  the  rvdtustness  of  the  ADM  network. 
Table  II  shows  that  under  the  second  rule,  the  ADM  net¬ 
work  is  verv  robust.  When  one  or  two  pairs  of  t/O  ports 
are  disabled  after  the  failure  of  a  link  or  a  switcoing  ele* 
ment,  respectively,  all  remaining  I/O  ports  are  unaffected. 
A  failure  can  eliminate  one  path  between  given  tinaflected 
input  and  output  ports,  but  routing  tag  methods  exist  for 
avoiding  such  fnuKs  by  using  nn  alternate  path  (as  in  the 
example  of  routing  from  input  0  to  output  5  above)  (24|. 
(A  method  for  improving  the  robustness  of  the  Generalised 
Cube  network  by  adding  one  extra  stage  has  been 
explored  in  |2j.) 

This  analysis  assumes  that  the  failure  of  one  com¬ 
ponent  is  independent  of  the  failure  of  any  other  com¬ 
ponent.  If  all  or  a  large  part  of  one  stage  is  implemented 
on  a  single  chip,  thb  assumption  may  or  may  not  be  valid. 
If  it  is  not,  then  the  networks  should  be  reanalyied  to 
account  for  the  new  failure  pattern  exhibited. 

To  exploit  the  robustness  of  any  network,  it  ia  impli¬ 
cit  that  faults  can  be  detected  and  diagnosed.  Such  eap» 
bilities  have  been  investigated  ia  |3,  14,  90, 34).  Dynamic 
fault  avoidance  has  been  diseuaaed  ia  (23, 24|. 


VI.  Coneluelona 

This  paper  has  examined  two  classes  of  muhistace 
interconnection  network  for  use  in  parnllel/dialr^tM 
systems;  the  cube  type  and  the  data  manipulator  type. 
This  was  done  by  comparing  a  representative  netwm 


from  each  rl.i.sR:  the  Generalised  Cube  and  the  Augmented 
Dal. 'I  Manipulator  (ADM).  From  a  straightforward 
c-ompariMin.  it  appears  that  the  ADM  network  is  more 
roNily,  but  more  ^werful.  This  paper  has  attempted  to 
quantify  the  differences  in  implementation  costs  by  consid¬ 
ering  comparable  implementation  models  for  both  net¬ 
works.  Furthermore,  using  a  graph  model  as  a  basis,  a 

3uanlilative  measure  of  comparative  robustness  was 
erived. 

Roferoneea 

|l]  C.  H.  Adams  III  and  II.  J.  Siegel,  "On  the  number  of 
permutations  performable  by  the  augmented  data 
manipulator  network,"  IEEE  Trana.  Comp.,  Vot.  C- 
.11.  pp  270.277.  Apr.  1082. 

|2]  C.  11.  .'\clams  III  and  H.  J.  Siegel,  "The  extra  stage 
cube;  a  fault-tolerant  interconnection  network  for 
super<y«lerns.'  IEEE  Trant.  Comp.,  Vol.  C-31,  pp. 
41.1-151,  M.ay  1082. 

|.lj  I).  IV  Agrawal,  "Testing  and  fault-tolerance  of  multis- 
tage  interconnection  networks,"  Computer,  Vol.  IS, 
pp.  4 1 -.5.1,  Apr.  1082. 

|l|  (•'.  Anderson,  and  B.  D.  Jensen,  "Computer  inter¬ 
connection  structures:  taxonomy,  characteristics,  and 
exninpl<>s,*  .-ICAf  Compufinf  Suruepa,  Vol.  7,  pp.  107- 
21.1.  Dec.  107.5. 

[.5|  G.  II.  Karnes  and  S.  F.  Lundstrom,  "Design  and  vali¬ 
dation  of  a  connection  network  for  many-processor 
multiprocessor  systems,”  Compnier,  Vol.  14,  pp.  31- 
41.  Dec.  1081. 

(6|  K.  K.  Batcher,  "STARAN  parallel  processor  ■•v.u»:.n 
hardware,"  AFIPS  1974  Ntir  r..,..p.  Coni,  M.  y 
1074,  pp.  40S-4I0. 

[7|  K.  1C.  Batcher,  "The  flip  network  ia  STARA.'s,"  1979 
InI'l.  Conf.  Parallel  Prot.,  Auf.  1976,  pp.  fiS-71. 

|8|  F.  Briggs,  et.  al..  "PUMPS  architeeture  for  pattera 
analysis  and  image  data-base  management,"  Prat. 
Patlern  Ret.  and  hmapt  Prot.  Conf.,  Aug.  1081,  pp. 
.T87-.108. 

[0|  L.  Ciminiera  and  A.  Serra,  “Modular  intcrconnectian 
networks  with  asynehronons  control,*  IJIh  Annual 
Uaraii  Ml.  Conf  SrL,  Jan.  1081,  pp.  210-218. 
jlO]  A.  M.  Despain  and  D.  A.  Patterson,  “X-tree:  a  tree 
structured  multi- processor  computer  architecture," 
51k  Annuaf  Ml.  Spmp.  Comp.  Arth.,  Apr.  1078,  pp. 
I44-I.5I. 

|ll]  D.  M.  Dias  and  J.  R.  Jump,  "Analysis  and  simulation 
of  buffered  della  networks,"  IEEE  Trono.  Comp.,  Vol. 
r-.10,  pp.  27.1-282,  Apr.  1081. 

|I2|  T.  Feng,  "Data  manipulating  functioiis  in  parallel 
prcM  CKsors  and  their  implementations,*  IEEE  Trono. 
Comp.,  Vol.  CC-23,  pp.  309-318,  Mar.  IW4. 

1 1.1)  T.  Feng.  “A  survey  of  interconnection  networks," 
Computer,  Vol.  14,  pp.  12-27,  Dec.  1981. 

|l  l)  T.  Feng  and  C.  Wu,  “Fault-diagnosis  for  a  class  of 
multistage  interconnection  networks,”  IEEE  TVans. 
Comp.,  Vol.  ('-.10,  pp.  743-758,  Oct.  1981. 

||.5|  G.  K.  Goke,  G.  J.  Lipovaki,  "Dnnyan  nctumrks  Ibr 
partitioning  multiproceanor  tystems,"  lot  Sgmp. 
Comp.  .Arrk.,  Dec.  1973,  pp.  21-28. 


920 


(16)  T.  Lug  and  II.  S.  Stoue,  “A  iiliufne>cx  change  nct- 
wf>rk  with  unipliiied  control,"  tHEE  Tratu.  Comp., 
Vol.  C>S$.  pp.  SS-ftS.  Ju.  1076. 

(17|  0.  II.  Lnwrie,  “Acccia  nnd  nlignment  of  dnu  in  an 
arrpy  procmor,”  IEEE  Tront.  Comp.,  Vol.  (%24,  pp. 
ll4VIISS,Dec.  167S. 

|I6|  M.  Klalek  and  W.  W.  Myre,  “A  dintcription  method  of 
interconnection  networks,"  IEEE  Teek.  Com.  Di$l. 
pTot.  Qaarferfp,  Vol.  1,  Feb.  1081,  pp.  1-6. 

(10)  W.  C.  McDonald,  J.  M.  Williams,  “The  advanced 
data  processing  test  bed,"  Compoot,  pp.  StO-SSI, 
Mar.  1078. 

|20)  K.  J.  McMillen,  Ph.D.  thesb,  in  preparation. 

joil  K.  J.  McMUIen,  G.  B.  Adams  III,  and  11.  J.  Siegel, 
"Performance  and  iroplemcntatiun  of  4x4  switching 
nodes  in  an  interconnection  network  for  PASM,"  ttSt 
M.  Conf.  PotoUel  Proreooinp,  Aug.  1081,  pp.  220-233. 

|22]  K.  J.  McMillen  and  H.  J.  Siegel,  “The  hybrid  cube 
network,"  Dutrikutti  Dolo  Acfnisilien,  CVtmpnliny, 
end  Control  Spmp.,  Dec.  1080,  pp.  11-22. 

(23|  R.  J.  McMillen  and  H.  J.  Siegel,  “Performance  and 
fault  tolerance  improvements  in  the  inverse  aug¬ 
mented  data  manipulator  network,”  9tk  Anniul  InI’l. 
Spmp.  Comp.  Artk.,  Apr.  1082,  pp.  63-72. 

|24|  R.  J.  McMillen  and  H.  J.  Siegel,  “Routing  schemes  for 
the  augmented  data  manipulator  network  in  an 
MIMD  system,"  KEE  Trono.  Comp.,  to  appear  Dec. 
1082. 

(25)  0.  S.  Parker  and  C.  S.  Raghavendra,  “The  Gamma 
network:  a  multiprocessor  network  with  redundant 
paths,"  WA  Annmol  h>%  Spmp.  Comp.  Artk.,  Apr. 
1482,  pp  73-80. 

'  J.  I '  ‘'at,^l,  “Perlorniancc  of  processor-memory  inter- 
c.’niicvtions  for  multiprocessors,"  IEEE  Tromo. 
Comp.,  Vol.  C-30.  pp.  771-780,  Oct.  1081. 

|27|  M.  C.  Pease,  "The  indirect  binary  n-cube  mieropro- 
cessor  array,”  IEEE  Trmu.  Comp.,  Vol.  C-28,  pp. 
450^473,  Mpy  ion. 

|28|  D.  K.  Prndhaa  and  K.  L.  Kodandapani,  “A  uniform 
representation  of  single-aad  multistage  interconnec¬ 
tion  networks  used  in  81MD  mnchines,”  IEEE  Tnmo. 
Comp.,  Vol.  C-20.  pp.  Tn-TOl,  Sept.  1080. 

|20|  U.  V.  Premkumar,  R.  Kapur,  M.  Malek,  G.  J. 
Lipovski,  P.  Horne,  "Design  and  implementation  of 
the  banyan  iatercaaneetion  network  in  TRAC,” 
APIPS  tSKNott  Comp.  Conf.,  June  1080,  pp.  043- 
058. 


I-'IOI  D.  D.  Ratbi  and  M.  Malek,  “Fault  diagnosis  of  net¬ 
works,"  Diotribulti  Doto  Aepmoition,  Compniinp,  end 
Control  Spmp.,  Dec.  1080,  pp.  110-110. 

{31]  M.  C.  Sejnowski,  E.  T.  Upchurch,  R.  N.  Kapur,  D.  P. 
S.  Charlu,  and  G.  J.  Lipovski,  “An  overview  of  the 
Texas  Rcconfigurable  Array  Computer,"  AFIPS  1980 
Not'l.  Comp.  Conf.,  June  1080,  pp.  631-641. 

(32|  H.  J.  Siegel,  "Interconnection  networks  for  SIMD 
ntaebines,"  Computer,  Vol.  12,  pp.  57-65,  June  1070. 

|33|  11.  J.  Siegel,  R.  J.  McMillen,  "Using  the  Augmented 
Data  Manipulator  Network  in  PASM,"  Cempuler, 
Vol.  14,  pp.  25-33,  Feb.  1081. 

(34)  II.  j.  Siegel  and  R.  J.  McMillen,  "The  multbtage 
cube:  a  versatile  interconnection  network,"  Computer, 
Vol.  14,  pp.  65-76,  Dee.  1081. 

(35|  II.  J.  Siegel,  R.  J.  McMillen,  P.  T.  Mueller,  Jr.,  "A 
survey  of  interconnection  methods  for  rcconfigurable 
parallel  processing  systems,"  AFIPS  1070  Null. 
Comp.  Conf,  June  l070,  pp.  520-542. 

(36)  II.  J.  Siegel,  L.  J.  Siegel,  F.  C.  Kemmerer,  P.  T. 
Mueller,  Jr.,  H.  E.  Smalley,  Jr.,  and  S.  D.  Smith, 
“PASM:  a  partitionable  SIMD/MIMD  system  for 
image  processing  and  pattern  recognitioa,"  IEEE 
Trent.  Comp.,  Vol.  C-30,  pp.  034-047,  Dec.  1081. 

|37]  H.  J.  Siegel,  S.  D.  Smith,  “Study  of  multbtage  SIMD 
interconnection  networks,"  5tk  Annmol  /all  Spmp. 
Comp.  Artk.,  Apr.  1078,  pp.  223-220. 

{38|  S.  D.  Smith,  “LSI  design  consideratioos  finr  multbtage 
interconnection  networks  for  parallel  processing  sys¬ 
tems,"  IJIA  Annuel /fnven  /nil  Conf.  Spi.  Set.,  Jan. 
1081,  pp.  210-227. 

(30]  R.  J.  Swan,  S.  11.  Fuller  and  D.  P.  Siewiorek,  “Cm*:  a 
modular,  multi-micrupioceasor,"  AFIPS  1077  Null. 
Pomp.  Conf,  June  1077,  pp.  637-644. 

(40)  K.  J.  Thurber,  “Interconnection  networks  -  a  survey 
and  assessment,"  AFIPS  1074  Nut!  Comp.  Conf., 
May  1074,  pp.  000-010. 

(41)  A.  R.  Tripathi,  G.  J.  Lipovski,  “Packet  switching  in 
banyan  networks,"  MA  Annuel  lull  5|pmp.  Cemp. 
ArcA.,  Apr.  1070,  pp.  160-167. 

(42)  L.  C.  Widdoas,  Jr.,  “The  Minerva  multi- 

microproccasur,"  drd  Annuel  Inti.  Spmp.  Cemp. 
ArcA.,  Jan.  1076,  pp.  34-30. 

('I3(  Wu,  T.  Feng,  “On  a  class  of  multbiage  intercon¬ 
nection  networks,"  IEEE  Tront.  Comp.,  Vol.  C-20, 
pp.  604-702,  Aug.  1080. 

(44)  W.  A.  Wulf,  C.  G.  Bell,  “C.mmp~a  multi- 
miniprocessor,”  AFIPS  1078  FoH  Joint  Comp.  Conf., 
Dec.  1072,  pp.  765-777. 


